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ABSTRACT 

This paper discusses the results of a study that 
investigated the progress of 51 children receiving long-term 
dynamically-oriented therapy at a naturalistic "real world" clinic. The 
orientation of the Outpatient Clinic of the Yale Child Study Center is 
primarily long-term, psychodynamic treatment. Parents and clinicians 
completed measures. before initiation of treatment and . at termination . Results 
indicated the children improved in their functioning as rated by parents and 
clinicians. In addition, parents reported improved family management 
practices and a decrease in social isolation from the beginning to the end of 
treatment. The paper discusses the difficulties inherent in evaluating 
effectiveness in naturalistic settings where clinicians are less motivated to 
adhere to research designs, especially those involving the administration of 
post measures. The need to conduct more outcome studies is urged. Findings 
for the study suggest that reliable and positive results can be achieved for 
those families and therapists in "real world" clinics who remain committed 
and accountable to the therapeutic process. (Contains 19 references.) (CR) 
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Introduction 



The majority of treatment outcome studies in outpatient children's mental health 
services have taken place in research clinics. Findings from research studies have 
been consistently positive. In contrast, there exist only a handful of published 
outcome studies based in "naturalistic" settings with mixed and far less positive 
results; Weisz and Weiss (1993) address this problem with the "good news bad 
news joke" which states that "the good news is that child psychotherapy works; the 
bad news is, not in real life" (p. 96). 

Contrasting procedures between research and "real world" clinics may explain these 
observed differences in treatment outcome. Werry and Andrews (1996) point out 
that "there is little resemblance between what researchers study and what 
practitioners do" (p. 879). Differences in procedures between the research and 
naturalistic settings include the following: subject recruitment, parental involvement 
in treatment, stringent treatment protocols, exclusionary criteria, manualized 
treatment, and completion of measures by both therapists and patients (Kazdin, 

1991; Kendall &. Southam-Gerow, 1995). Such procedures are often an anathema to 
both clients and clinicians in naturalistic settings and likely result in different 
clinicians treating and different subject populations being seen at research versus 
non-research based clinics. 
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In the adult literature, Seligman (1995) describes this dialectic by distinguishing 
between "efficacy" studies in research clinics and "effectiveness" studies in 
"naturalistic" settings. Under tightly controlled conditions with sophisticated 
1 methodology, efficacy studies conducted in research clinics differ from 
effectiveness studies which assess "how patients fare under the actual conditions of 
treatment in the field" (Seligman, 1995, p. 966). Although both approaches 
contribute to our knowledge of treatment outcome, "effectiveness" studies may hold 
I the greatest potential to teach us about the majority of clients and clinicians who 
participate in the "real world" of mental health. Seligman (1995) further extends this 
contention by asserting that "the efficacy study is the wrong method for empirically 
validating psychotherapy as it is actually done" (p. 966). However, a number of 
investigators in children's mental health have proposed research methods to close 
this chasm. They have underscored the need to examine ways to improve the 
transition from efficacy research to effectiveness research (Clark, 1 995), increase 
the transportability of research based treatment to naturalistic settings (Kendall &. 
Southam-Gerow, 1995), and bridge the gap between laboratory /university and 
clinic/community based treatment (Henggeler, Schoenwald, &. Pickrel, 1995; 
Hoagwood, Hibbs, Brent, &. Jensen, 1995; Weisz, Donenberg, Han, &. Weiss, 1995). 

The present study was designed to contribute to the scant literature on outcome data 
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derived from naturalistic settings. It was hypothesized that children would 
demonstrate reliable and positive improvement from pre- to post-treatment based on 
standard parent and therapist outcome measures. Specifically, parent rated 
improvement of their children's internalizing and externalizing symptoms on the 
Child Behavior Checklist (CBCL; Achenbach & Edelbrock, 1 983) and therapist 
rated improvement on both the Global Assessment of Functioning (GAF; Diagnostic 
and Statistical Manual of Mental Disorders, 1987) and the Children's Global 
Assessment Scale (CGAS; Shaffer et al., 1983) were expected. Furthermore, given 
an emphasis in treatment on parent involvement, it was hypothesized that families 
would report higher levels of family cohesion and organization, less family conflict, 
and a greater sense of community involvement (e.g., recreation) as assessed by the 
subscales of the Family Envirorunent Scale (FES; Moos & Moos, 1986). 

Method 

Subjects 

Parents and clinicians of fifty-one children (N = 51; 32 boys/ 19 girls), receiving 
between 1 6 and 90 therapy sessions, completed measures before initiation of 
treatment and at termination. All cases represent agreed-upon discharge between the 
family and therapist. Demographic variables are presented in Table 1 . This sample 
represents a fraction (less than 5%) of the clinic population over a 5-year period 
(1989-1994) and contains an over- representation of Caucasian children (80%), 
cases that were self-referred (94%), and cases with significantly more treatment 
sessions (M = 43) in comparison to our general clinic population. During the time 
period that this data was collected, Caucasian children represented 40% of the clinic, 
only 20% of the children were self-referred, and cases were seen an average of 24 
sessions. However, the current subject sample did not differ from the general clinic 
pool from which it came in age, gender, caretaker status (one parent versus two 
parent), residence (urban versus suburban), socioeconomic status as measured by the 
Hollingshead (1975) Five-factor Index of Social Status, or pay source (insurance 
versus medicaid). 

Measures 

The parent rated CBCL includes 1 1 8 items on a 0-2 point scale which rate a variety 
of behavior problems and yields an internalizing, externalizing, and total score 
composite. The FES includes 90 true-false items about family functioning across 10 
subscales (i.e., conflict, cohesion, expressiveness, independence, achievement, 
intellect, religious, recreation, organization,‘&i:ontrol). The GAF measures overall 
adaptive functioning and is rated by clinicians on a 1 00 point scale (higher scores 
indicate better functioning). Similarly, the CGAS is rated by clinicians on a 100 
point scale and measures the child's lowest level of functioning within the past week 
(higher scores indicate better functioning). 

Procedures 

All clients were seen in the Outpatient Clinic of the Yale Child Study Center. The 
clinic's orientation is primarily long-term, psychodynamic treatment. Prior to clinic 
contact, parents completed both the CBCL and FES; these two measures were also 
completed by parents at discharge. The treating clinician rated the child on the GAF 
and CGAS at evaluation and discharge. 

Results 

Main Analyses 



To test our general hypothesis that children would improve as a result of therapy, we 
ran paired sample t-tests on our primary criterion measures of interest (i.e., the 
parent completed CBCL and FES scales and the clinician rated GAF and CGAS). In 
order to control for chance findings, we used the conservative Bonferonni correction 
and required a value ofp = .006 (i.e., .05/8) to consider our findings significant. At 
this adjusted alpha level, significant improvement was reported on the CBCL 
(internalizing, externalizing, and total scales), the CGAS, and GAF. In addition, 
there was a significant increase in the "Recreation" and "Organization" subscales of 
the FES (see Table 2) . There was no significant change from pre- to post-treatment 
on the "Conflict" and "Cohesion" subscales of the FES. 

Discussion 

Fifty-one children receiving long term dynamically oriented therapy at a "real 
world" clinic improved in their functioning as rated by parents and clinicians. In 
addition, parents reported improved family management practices and a decrease in 
social isolation from the beginning to end of treatment. These parallel gains in child 
and family functioning address the interactions between child and family well-being 
and are consistent with prior research (Armbruster, Dobuler, Fischer, & Grigsby, 
1996). 

Despite these positive results, the validity of our findings is limited by a number of 
factors. First, without a control group it is difficult to attribute improvement to the 
intervention or to natural maturational processes. The effect of maturation is 
particularly relevant in view of an average treatment duration of nearly a year. 
Second, although the sample was similar to the general clinic population in age, 
gender, caretaker status, residence, pay source, and SES, it differed in that it was 
comprised primarily of Caucasian, self-referred, long term therapy cases. Finally, 
only families who were discharged from treatment by mutual agreement between 
therapist and client and completed the post measures were included in the study. 
Excluded were those families who dropped out of treatment or were discharged but 
who did not complete the post measures. Follow-up for these families was beyond 
the scope of this study. 

These limitations may be understood in a number of ways. The preponderance of 
Caucasian families in this study may point to the issue of cultural congruity between 
therapist and client (Armbruster & Kazdin, 1994). Studies have shown that similar 
ethnicity/race of clinician and client predict continuance in therapy (Cheung & 
Snowden, 1990; Flaskerud, 1986). At the time of this study, there was a significant 

lack of minority therapists at the clinic which may have interfered with“African 

American families forming positive alliances with their therapists. The group for 
which pre- and post- measures were collected may reflect a "goodness of fit" (Chess 
& Thomas, 1986) between family and therapist. This "fit" may be partially 
accounted for by compatible levels of motivation, accountability and commitment, 
as well as similar racial/ethnic characteristics. The motivational factor is further 
supported by the fact that the majority (94%) of families were self-referred. 

In this study, self-selection bias in clinician and parent may lead us to conclude that 
treatment is effective, when in fact, we haye sampled only a small percentage of 
highly motivated parents and clinicians. This study underscores difficulties inherent 
in evaluating effectiveness in "naturalistic" settings where clinicians are less 
motivated to adhere to research designs, especially those involving the 
administration of post measures (Henggeler et al., 1995; Hoagwood et al., 1995; 
Kazdin, 1991). The advent of managed care with its requirements for accountability 
(e.g., treatment outcome data) may provide the opportunity to conduct research in 
"naturalistic" settings with a more representative sample. In addition, managed care 
will undoubtedly restrict the length of treatment and reduce the likelihood that 
improvement is due solely to maturational effects. Future studies will be able to 
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improvement is due solely to maturational effects. Future studies will be able to 
examine populations within this modified framework. 

In summary, "naturalistic" (Weisz & Weiss, 1993) or "real world" (Jensen, Bloedau, 
& Davis, 1990) child outpatient mental health clinics represent the majority of 
mental health services provided to children. Our findings suggest that reliable and 
positive results can be achieved for those families and therapists in "real world" 
clinics who remain committed and accountable to the therapeutic process. Although 
this is an initial study with limitations, it offers an optimistic glimpse into the 
outcomes achieved by "treatment as usual." 
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